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ABSTRACT 

No Child left Behind, The ESEA Flexibility Program, and the Race to the Top contest 
have spurred a multitude of School Improvement programs, businesses, initiatives and more. 
The current emphasis on standardized testing, data gathering and analysis, and measuring 
effective schooling has caused many education agencies, state and local, to seek evaluation 
services and to employ evaluation methods. A Guided Empowerment Self-Audit of any 
organization can reveal areas of concern that the organization can address to improve the 
effectiveness of the organization. The Guided Empowerment Self Evaluation approach is 
designed to help school communities monitor and evaluate their own performance while using 
indicators for School Improvement and School Turnaround. Anytime the people in the 
organization can be involved in a meaningful self-analysis and development of a plan of action, 
they are more apt to take ownership in the plan and carry it through to fruition. A Guided 
Empowerment Self-Audit Evaluation builds program capacity and fosters program 
improvement. It teaches people to help themselves by learning how to evaluate their own 
programs. Key concepts include: a critical friend, cycles of reflection and action, and a 
community of learners. The basic steps of empowerment evaluation include: 1) establishing a 
mission or unifying purpose; 2) taking stock - creating a baseline to measure growth and 
improvement; and 3) planning for the future - establishing goals and strategies to achieve 
objectives, as well as credible evidence to monitor change. The Guided Empowerment Self- 
Audit uses the Eleven Identified Indicators of Effective Schooling identified by an international 
evaluation firm. This research project was a comparison of Guided Empowerment Self-Audits 
performed by 8 schools to determine common areas of concern uncovered by the schools. One 
of the major areas of concern revealed by the Guided Empowerment Self-Audits was a 
disconnect between teachers’ perceptions of the implementation of the Eleven Indicators of 
Effective Schooling and the school’s student achievement data. Identifying barriers to success 
can aid any organization as it seeks to improve. 

Keywords: Empowerment, Self-Audit, School Improvement 


Copyright statement: Authors retain the copyright to the manuscripts published in AABRI 
journals. Please see the AABRI Copyright Policy at http://www.aabri.com/copyright.html . 


A guided empowerment self-audit, page 1 



Research in Higher Education Journal Volume 25 - September, 2014 


INTRODUCTION 

Various strategies have been proposed in recent literature to turn around student 
achievement in schools since the implementation of the Race to the Top Initiative. Strategies to 
improve student achievement abound, especially in low performing schools. These schools have 
been analyzed by various external methods to determine the root cause of their lack of 
performance or mediocre performance. Educators have looked to the business world for 
strategies that may also be used in education. One such strategy is the empowerment evaluation. 
Empowerment Evaluation was conceptualized and initially introduced at the American Evaluation 
Association conference by David Fetterman. The seminal work on empowerment evaluation was 
completed in 1996 by Fetterman, Kraftarian, and Wandersman. Empowerment evaluation, 
according to David Fetterman, (2007), is: 

The use of evaluation concepts, techniques, and findings to foster improvement and self- 
determination. It employs both qualitative and quantitative methodologies. Although it 
can be applied to individuals, organizations, communities, and societies or cultures, the 
focus is usually on programs. Empowerment evaluation is a part of the intellectual 
landscape of evaluation. A wide range of programs use EE, including substance abuse 
prevention, indigent health care, welfare reform, battered women's shelters, adolescent 
pregnancy prevention, individuals with disabilities, doctoral programs.” Further, 
descriptions of programs that use empowerment evaluation appear in Empowerment 
Evaluation: Knowledge and Tools for Self-Assessment and Accountability (Fetterman, 
Kaftarian, & Wandersman, 1996). 

In addition, this approach has been institutionalized within the American Evaluation 
Association and is consistent with the spirit of the standards developed by the Joint Committee 
on Standards for Educational Evaluation (Fetterman, 1995b; Joint Committee, 1994). Most 
recently empowerment evaluations have been used in K-12- public schools as a tool for school 
improvement initiatives. In 2001, the Arkansas Department of Education, Division of 
Accountability, selected David Fetterman to conduct Empowerment Evaluations on school 
districts identified in Academic Distress in order to help those schools identify areas that 
required immediate improvement (Fetterman & Smith 2001). 

What is the Value of an Empowerment Self-Audit Evaluation? 

Anytime the people in the organization can be involved in a meaningful self-analysis and 
development of the plan of action, they are more apt to take ownership in the plan and carry it 
through to fruition. When schools participate collaboratively and actively in an Empowerment 
Evaluation, they are more likely to make decisions and take actions based on their evaluation 
data. The most current definition of Empowerment Evaluation is: “An evaluation approach that 
aims to increase the probability of achieving program success by (1) providing stakeholders with 
tools for assessing the planning, implementation, and self-evaluation of their program, and (2) 
mainstreaming evaluation as part of the planning and management of the program/organization.” 
(Wandersman et al., 2005, p.28). 

Prior to Empowerment Evaluations or Self Evaluations, organizations have traditionally 
hired an independent evaluator to conduct an evaluation of their strategies for them (Rossi, 
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Freeman & Lipsey, 1999). In some cases these evaluations were not used because stakeholders 
did not have any ownership in the report or the process. Unlike the traditional external 
evaluation, whose evaluators often worked hard to understand and develop an evaluation plan to 
address a school’s concerns and improve, in the Empowerment Self-Evaluation, the school 
leadership with first- hand knowledge of the school and the entire staff become active 
participants in the evaluation process (Cox, Keener, Wandersman A 2009). 

According to David Fetterman (2007), a pragmatic influence on empowerment 
evaluation is the W. K. Kellogg Foundation's emphasis on empowerment in community settings. 
The foundation has taken a clear position concerning empowerment as a funding strategy: 
Empowerment evaluation is largely based on Zimmerman’s theory of empowerment (Fetterman, 
1996). Zimmerman suggests that, 

“We've long been convinced that problems can best be solved at the local level by people 
who live with them on a daily basis. In other words, individuals and groups of people 
must be empowered to become changemakers and solve their own problems, through the 
organizations and institutions they devise.... Through our community-based 
programming, we are helping to empower various individuals, agencies, institutions, and 
organizations to work together to identify problems and to find quality, cost-effective 
solutions. In doing so, we find ourselves working more than ever with grantees with 
whom we have been less involved—smaller, newer organizations and their programs. 

(' Transitions, 1992, p. 6) 

A review of the literature suggests that interest in self-evaluations in schools is growing. 
School self-evaluation can be a fundamental force in achieving school improvement (CFBT 
Education Trust, 2013). The real value in Empowerment Evaluations can best be described by 
educators, such as Samuel White, principal of the Brinkley High School who suggested that, 
“finally we have reports that we can use.” (May 1, 2013). 

What is the Empowerment Evaluation Approach to School Improvement? 

The Guided Empowerment Self-Audit Evaluation approach is designed to help school 
communities monitor and evaluate their own performance while using local, state and USDOE 
indicators for School Improvement and School Turnaround Principles as benchmarks. It is used 
in comprehensive educational initiatives and is designed to help groups accomplish their goals. 
“Empowerment Evaluation” is the use of evaluation concepts, techniques, and findings to foster 
improvement and self-determination" (Fetterman, 2007). The Guided Empowerment Self-Audit 
Evaluation Approach uses Empowerment Coaches as critical friends to help schools conduct the 
internal self evaluation and to check initial perception against actual school achievement data. 

What is the Role of the Guided Empowerment Self-Audit Team/Critical Friends or 
Coaches? 

Guided Empowerment Self-Audit Team/Critical Friends or Coaches help the school staff 
conduct the self-evaluation. Coaches generally provide general guidance and direction to the 
effort, attending sessions to monitor and facilitate as needed. It is critical to emphasize that the 
staff are in charge of their effort; otherwise, program participants initially tend to look to the 
empowerment evaluator as expert, which makes them dependent on an outside agent. The 
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Empowerment Team meet with parents, teachers, students and community members as they 
conduct the evaluation to provide support and help the staff review current evidence. The 
Empowerment Evaluation Team clears away obstacles and identifies and clarifies 
miscommunication patterns. They participate in meetings along with internal empowerment 
evaluators, providing explanations, suggestions, and advice at various junctures to help ensure 
that the process has a fair chance and that perceptions are checked against current school 
achievement data. 

An Empowerment Evaluation Coach can also provide useful information about how to 
create facilitation teams (balancing analytical and social skills), work with resistant (but 
interested) units, develop refresher sessions to energize tired units, and resolve various protocol 
issues. Simple suggestions along these lines can keep an effort from backfiring or being seriously 
derailed. A coach may also be asked to help create the evaluation design with minimal additional 
support. The Empowerment Evaluation Coach ensures that the evaluation remains in the hands 
of program personnel. The coach's task is to provide useful information, based on the evaluator's 
training and past experience, to keep the effort on course. 

How is the Guided Empowerment Self Audit Used in Schools? 

The Empowerment Evaluation process is designed to be used by people. It places 
evaluation in the hands of the school community and staff members. The more that people are 
engaged in conducting their own evaluations, the more likely they are to believe in them because 
the evaluation findings are theirs. In addition, a byproduct of this experience is that school 
leaders learn to think evaluatively. The Guided Empowerment Self-Audit is a process by which 
members of staff in a school enter cycles of reflections on their current practice, check current 
perceptions against established evidence and identify areas for action to stimulate improvement 
in the areas of school performance and improved educator capacity. 

Feedback received from teachers, principals, students, and staff members are generally 
extremely positive; and all staff actively engage in the empowerment process. Often initial staff 
perceptions are changed as schools explore data to ensure that initial perceptions can be 
validated. Rather than an external evaluator presenting an independent report, school personnel 
are guided by self- exploration and inquiry. During the process, new ideas and subtle changes 
can be implemented immediately implemented suggestions that could impact immediate wins. 
Therefore, the Empowerment Evaluation report and empowering self analysis process will be 
used collaboratively for school improvement planning as the school continues to make progress. 

According to Miller and Campbell, (2006) and Fetterman (2007), There are ten principles 
that are an essential means for guiding empowerment evaluations. These principles are suggested 
to be associated with the process of an empowerment evaluation. They include: 1. Improvement 
2. Organizational Learning 3. Community Ownership 4. Inclusion 5. Democratic participation 6. 
Social Justice 7. Community knowledge 8. Evidence-based strategies 9. Capacity building 10. 
Accountability. 

Schools use Empowerment Self-Audit findings to strengthen their internal teaching and 
learning systems and thereby facilitate the improvement of the quality of their core academic 
activities. Decisions about the manner in which this is done, and the priority accorded to the 
various recommendations are collaboratively planned with district personnel. It is important to 
note that program participants—including clients—conduct their own evaluations; an outside 
Empowerment Evaluation Coach serves as an additional facilitator depending on internal 
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program capabilities 

METHODOLOGY OF THE GUIDED EMPOWERMENT SELF-AUDIT 

The Guided Empowerment Self-Audit Evaluation is conducted as a qualitative and quantitative 
research study encompassing the basic research steps of: 

1. Posing a research question. 

The question for the study is, “What is the state of your school as perceived by 
the stakeholders, how does that perception compare with the data, and what 
should your school do from this point to improve student achievement? 

2. Collecting qualitative data: Surveys, Interviews, Artifacts, & Focus Groups. 

3. Triangulating the data with student achievement data to answer the research question. 

4. Drawing Conclusions. 

5. Creating recommendations for change. 

Three critical decisions are made before the data collection begins: 1.) What criteria will 
be used as the framework for organizing the data? 2.) How will the data be analyzed? 3.) What 
evaluative measure will be used to assign levels of implementation on the established criteria? 

First, a review of the blended set of school performance indicators encompassing the 
State, local school district and the Federal School Turnaround Principles is conducted. Second, 
a decision is made to triangulate the data from document perusals, interviews, observations and 
perception surveys. Third, a Common Rubric is adopted with a 3 point Likert Scale representing 
the following categories: 

1. Needs Improvement 

2. Acceptable 

3. Exemplary. 

Identifying the Eleven Indicators of School Performance 

National, State and local districts have established standards of school performance. For 
example in Arkansas, the Nine Arkansas Scholastic Audit Standards are used as the measuring 
instrument for Arkansas Schools’ performance. To be customized and inclusive, the National 
Seven Principles of School Turnaround are also combined to ensure a complete examination 
based on the dual research behind these two measures of school performance. Schools can 
perform a crosswalk between the local and state standards with the National standards. A sample 
cross-walk of the two rubrics is identified as The Eleven Indicators of Effective Schooling. 

The following table denotes the final Eleven Indicators of Effective Schooling agreed 
upon as a result of combining of the ADE Scholastic Audit Standards and the Federal Seven 
Principles of School Turnaround. 
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Table 1: The Eleven Indicators of Effective Schooling 


ADE Scholastic Audit 
Indicators/ Arkansas 
School improvement 
Standards 

Federal Seven 
Principles of School 
Turnaround 

The Eleven Indicators of Effective 
Schooling 

Academic Performance 
Standard 1: Curriculum- 
The school develops and 
implements a curriculum 
that is rigorous, intentional, 
and aligned to state and 
local standards. 

College and Career ready 
standards and curriculum 

Academic Performance 

Indicator 1: Curriculum-The school 
develops and implements a curriculum that is 
rigorous, intentional, aligned to Common 

Core, and state and local standards. 

Academic Performance 
Standard 2: Assessment- 

The school utilizes multiple 
evaluation and assessment 
strategies to continuously 
monitor and modify 
instruction to meet student 
needs and support proficient 
student work. 


Academic Performance 

Indicator 2: Assessment-The school utilizes 
multiple evaluation and assessment strategies 
to continuously monitor and modify 
instruction to meet student needs and support 
proficient student work, teachers are trained 
in assessment literacy. 

Academic Performance 
Standard 3: Instruction- 

The school’s instructional 
program actively engages all 
students by using effective, 
varied, and research-based 
practices to improve student 
academic performance. 

Step 1 High Quality 
Teaching: Ensuring that 
teachers are effective and 
able to improve 
instruction. Maintain a 
consistent focus on 
improving instruction. 

Academic Performance 

Indicator 3: Instruction/High Quality 
Teaching-The school’s instructional program 
actively engages all students by using 
effective, varied, and research-based practices 
to improve student academic performance, 
and teachers are effective and able to improve 
instruction. 

Learning Environment 
Standard 4: School 
Culture-The school/district 
functions as an effective 
learning community and 
supports a climate 
conducive to performance 
excellence. 

Step V. School 
Environment: 

Establishing a school 
environment that 
improves student 
achievement, and 
student’s needs 

Learning Environment 

Indicator 4: School Culture-The 
school/district functions as an effective 
learning community, improves student 
achievement, and supports a climate 
conducive to performance excellence. 

Learning Environment 
Standard 5: Student, 

Family, and Community 
Support-The school/district 
works with families and 
community groups to 
remove barriers to learning 
in an effort to meet the 
intellectual, social, career, 
and developmental needs of 
students. 

Step IV. School 
Community Engagement 

Learning Environment 
indicator 5: Student, Family, and 

Community Support-The school/district 
works with families and community groups to 
remove barriers to learning in an effort to 
meet the intellectual, social, career, and 
developmental needs of students. 
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ADE Scholastic Audit 
Indicators/ Arkansas 
School improvement 
Standards 

Federal Seven 
Principles of School 
Turnaround 

The Eleven Indicators of Effective 
Schooling 

Learning Environment 
Standard 6: Professional 
Growth, Development, and 
Evaluation-The 
school/district provides 
research-based, results 
driven professional 
development opportunities 
for staff and implements 
performance evaluation 
procedures in order to 
improve teaching and 
learning. 


Learning Environment 

Indicator 6: Professional Growth, 
Development, and Evaluation-The 
school/district provides research-based, 
results driven professional development 
opportunities for staff and implements 
performance evaluation procedures in order to 
improve teaching and learning. 

Efficiency 

Standard 7: Leadership- 

School/district instructional 
decisions focus on support 
for teaching and learning, 
organizational direction, 
high efficiency performance 
expectations, creating a 
learning culture, and 
developing leadership 
capacity. 

Step II: Effective 

School Leadership 

Providing strong 
leadership by a track 
record in improving 
achievement and has the 
ability to lead the 

turnaround pffort 

Efficiency 

Indicator 7: Leadership-School/district 
instructional decisions focus on support for 
teaching and learning, School Turnaround, 
high efficiency performance expectations, 
creating a learning culture, and developing 
leadership capacity. 

iui iicii u uiiu Liivji t 


Efficiency 

Standard 8: 

Organizational Structure 
and Resources-The 
organization of the 
school/district maximizes 
use of time, all available 
space and other resources to 
maximize teaching and 
learning and support high 
student and staff 
performances. 


Efficiency 

Indicator 8: Organizational Structure and 
Resources-The organization of the 
..school/district maximizes use of time, all 
available space and other resources to 
maximize teaching and learning and support 
high student and staff performances. 

Efficiency 

Standard 9: 

Comprehensive and 
Effective Planning-The 

school/district develops, 
implements, and evaluates a 
comprehensive school 
improvement plan that 
communicates a clear 


Efficiency 

Indicator 9: Comprehensive and Effective 
Planning-The school/district develops, 
implements, and evaluates a comprehensive 
school improvement plan that communicates 
a clear purpose, direction, and action plan 
focused on teaching and learning. 


A guided empowerment self-audit, page 7 











Research in Higher Education Journal Volume 25 - September, 2014 


ADE Scholastic Audit 
Indicators/ Arkansas 
School improvement 
Standards 

Federal Seven 
Principles of School 
Turnaround 

The Eleven Indicators of Effective 
Schooling 

purpose, direction, and 
action plan focused on 
teaching and learning. 



Use of Data Principle 

Step IV: Collaborative 
use of Data: Using data 
to inform instruction and 
for continuous 
improvement by 
providing time 
collaboration on the use 
of data 

Efficiency 

Indicator 10: Use of Data, Assessment 

Using data to inform instruction and for 
continuous improvement by providing time 
collaboration on the use of data. 

Extended Day Principle 

Step VII. Extended 

Day/ Extended Year 
Activities 

Efficiency 

Indicator 11: Extended Day Principle 

Extended learning is provided to enhance 
academic and non-academic skills. 


Triangulation of Data 

The Empowerment Self Audit report is based on a triangulation of data gathered from 
perusal of documents, observations, interviews, and completed perceptual survey results as 
depicted in Figure 1 below. 


Figure 1: The Triangulation of Data 
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The Common Rubric 

In order to simplify results and create a more cohesive understanding, the following 
rubric is used for documenting opinions on the implementation of the Eleven Indicators of 
Effective Schooling. The Common Rubric is explained in Figure 2. 

Figure 2: Self Audit Performance Levels 



Table 2: Common Rubric and Benchmarks 


3 = Exceptional/Exemplary 

2 = Acceptable 

1 = Needs 

Revision/Improvement 

Clear evidence is presented to 
show the current level of 
implementation of the 
indicators assessed in this 
category. For each indicator 
assessed as fully implemented 
there is clear evidence that it 
has become an established 
practice in the school and is 
sustainable over time. 

Some evidence is presented to 
show the current level of 
implementation of the 
indicators assessed in this 
category. For each indicator 
assessed as fully implemented 
there is some evidence that it 
has become an established 
practice in the school. 

Little or no evidence is 
presented to show the current 
level of implementation of the 
indicators assessed in this 
category. For each indicator 
assessed as fully implemented 
there is little or no evidence 
that it has become an 
established practice in the 
school. 

Created tasks for this category 
represent a concise focus for 
improvement and clearly 
demonstrate the capacity for 
achieving full implementation 
by target dates based on 
available resources. Strategies 
are clear and likely to increase 

Created tasks for this category 
represent some focus for 
improvement. Demonstration 
of capacity for full 
implementation by targets 
dates is stated but may not be 
realistic based on available 
resources. Strategies are 

Created tasks for this category 
are not evident or not realistic 
which demonstrates a 
perceived inability to 
successfully implement. 
Strategies have not been 
provided, or it is not clear how 
strategies will increase the 
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3 = Exceptional/Exemplary 

2 = Acceptable 

1 = Needs 

Revision/Improvement 

the quality of instruction, using 
research-based methods and 
strategies. 

mostly clear and may increase 
the quality of instruction, using 
research-based methods and 
strategies. 

quality of instruction, using 
research-based methods and 
strategies. 

It is apparent that the school 
leadership team has 
continuously worked toward 
completion of tasks, adding 
new tasks and new indicators 
throughout the year. Clear 
evidence is presented that 
completed indicators have 
become established practices 
in the school and are 
sustainable over time. 

It is apparent that the school 
leadership team has 
continuously worked toward 
completion of tasks for the 
planned indicators. Clear 
evidence is presented that 
completed indicators have 
become established practice in 
the school. 

There is little or no evidence 
that planned indicators and 
tasks have been continuously 
monitored. 




DATA COLLECTION 


Overview of the Data Collection Process 

The Empowerment Self-Audit process is conducted in three distinct phases. First, 
activities are performed before the On-site Visit, such as planning, gathering the Portfolio 
Documents, and researching the statistical information about the school. 

Second, the On-site visit is conducted over a two day visit. On Day One, the 
Empowerment panel coaches arrive at the school to observe the morning activities. As the 
instructional day begins, the panel conducts an introductory meeting with the principal and the 
school’s leadership team. 

Next Empowerment Self-Audit panel members, in conjunction with the principal, 
conduct a benchmarking observation for inter-rater agreement in a principal selected classroom. 
The Empowerment Self Audit panel members later conduct classroom observations, interviews 
with teachers and students and observe routines and procedures. Additionally Focus group 
interviews are conducted with groups of students, parents, and teachers throughout the day. 

At the end the instructional day on Day One of the visit, the Empowerment Self-Audit 
Panel conducts a “Taking Stock” meeting with the school faculty followed by a personal cycle of 
reflection with the principal analyzing the “Taking Stock” activity results and the critical role it 
plays in the School Improvement process. While the principal and A Guided Empowerment Self- 
Audit panel conducts the reflection over the Taking Stock results, the faculty proceeds to 
complete on-line perception surveys on Classroom Expectations and Teacher Perceptions. 

On the second day of the On-Site Visit, the panel again arrives and begins observations of 
the “routines and procedures” in the building. The principal and the Guided Empowerment Self 
Audit Panel conduct a second benchmarking observation for inter-rater agreement, in another 
principal selected classroom. Similar to Day One activities, the audit panel continues with 
evidence gathering in the form of classroom observations, interviews, and hallway and procedure 
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observations. 

Also on Day Two, parents and students complete on-line perception surveys linked to the 
indicators on the Scholastic Audit and the Seven Principles of Turnaround. Day Two ends with 
an exit interview with the principal. After the on-site visit, The Empowerment Self Audit Panel 
organizes, triangulates, and analyzes the collected data that leads to the finished report. 

Detailed Explanation of the Self-Audit Activities 

Prior to the on-site visit, the principal and faculty collect appropriate artifacts for the 
Audit Portfolio which included evidence such as Annual School Performance documentation, 
previous Scholastic Audits, School Profile, curriculum maps, pacing guides, PLC Meeting 
Minutes, parent meetings, Extended Day Organization, and schedules. 

DURING THE ON-SITE VISIT: DAY ONE 


During the audit visit, the Audit Panel conducts the activities in a professional manner. 
To set the tone for the on-site visit, A Guided Empowerment Self-Audit Panel begins the On¬ 
site visit by sharing an adopted definition of excellence. Namely, “Excellence is the result of 
caring more than others think is wise, risking more than others think is safe, dreaming more than 
others think is practical, and expecting more than others think is possible.” ~ Mac Anderson 
During the visit, the panel conducts awareness sessions with the principal, assistant 
principal, and leadership team members. The awareness sessions that include introductions, an 
overview of the audit process, and establishment of an open door policy. Teachers are asked to 
think evaluatively about the high expectations for student performance by Arne Duncan. 

Figure 3 
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Step 1: Introductory Meeting with the Principal 

The Guided Empowerment Self-Audit panel conducts an introductory meeting with the 
principal, in order to establish common expectations for the two-day visit. During this time, the 
advantages and the overview of activities of the Guided Empowerment Self-Audit are discussed 
with the principal. The principal provides the panel with the Portfolio: school profile, school 
maps, and names of staff members. The team notifies the principal how each staff member will 
be able to participate in the Guided Empowerment Self-Audit process during the empowerment 
session at the end of the school day. 

Figure 4: Empowerment Self Audit Advantages 



Step 2: Guided Empowerment Self-Audit Awareness Sessions with School Leadership 
Team 

During the opening awareness session the Empowerment Self-Audit Panel presents the 
Self-Audit process in detail with information pertinent to the topics listed below: 

1. Overview of the self-audit 

2. The data triangulation process used through the self-audit (Figure 1) 

3. Essential questions 

4. Interview protocol 

5. Requested school artifacts 

6. Self-audit advantages 

7. Standards and indicators for school improvement (Scholastic Audit) 

8. Seven Principles of School Turnaround 

9. Common rubric used to measure performance (Figure 2) 

10. Methodology 

11. Data collection process and data sources 

12. Classroom observations 
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13. Items observed during observation including teacher focus, direct instruction, education 
monitoring, procedures, and academic modeling 

14. Bloom’s Taxonomy 

15. Instruction strategies 

16. Assessment 

17. Learner Engagement 

18. Learning Environment 

19. Checklists 

20. Perception surveys 

21. Triangulation of finding 

22. Summary 

Members of the leadership team at the school are provided the opportunity to ask questions, 
provide additional documentation, and are given appropriate feedback during cycles of 
reflection. 


Step 3: Classroom Benchmarking for Inter-rater Agreement 

To ensure inter-rater agreement, the panel members and the principal have a common 
understanding and expectations for classroom observation data collection, a benchmarking 
process is conducted so that the principal can participate and better understand how classrooms 
will be rated and monitored. The benchmarking process helps to establish the degree of 
agreement among raters. According to Gwet, K. L. (2010) " Handbook of Inter-Rater Reliability 
(2nd Edition) the process “gives a score of how much homogeneity , or consensus , there is in the 
ratings given by judges. It is useful in refining the tools given to human judges, for example by 
determining if a particular scale is appropriate for measuring a particular variable. 

Figure 5: Classroom Observations 




Classroom Observation 


□To view the entire school, not to 
evaluate individual teachers 
□To gather data unassumingly 
□To serve as a catalyst for reflective 
school-wide dialogue 
□To help educators articulate and 
practice customary, attentive teaching 
that enhances student learning 



The following steps are employed in the benchmarking process: 


1. All team members monitor the classroom during the session against an established 
standard. 

2. A rating is given to the classroom observed. 

3. The Raters share and compare their assigned ratings through observation and feedback 
with each member of the team. 

4. An agreement on the rating is established. 
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5. Rating expectations for the rest of the classes are discussed with the principal. 

Step 4: The On-Site Work of the Audit Panel 

The Guided Empowerment Self-Audit Panel is split into three groups with assignment 
for: (1) classroom observations, (2) interviews, and (3) documentation reviews. The panel 
members rotate responsibility for these roles throughout the visit. 

An “Open Door Policy” is established and available for any staff or student member of 
the school as well as any members of the community to meet the Guided Empowerment Self- 
Audit Panel and make a submission. Interviews are completed and verbal feedback is given to 
the Principal while keeping the names of the interviewees confidential. The Panel interviews 
stakeholders from the 9 different groups listed below. 

1. Parent Focus Groups 

2. Individual Parents 

3. Leadership Team Members and Principal 

4. Administrative Staff Members 

5. Support Staff 

6. Student Focus Groups 

7. Individual Students 

8. Civic and Community Representatives, (when available) 

9. Teachers 

Step 5: On-going Critical Coaching/Cycles of Reflections 

The Guided Empowerment Self-Audit Panel provides evidence-based feedback, 
recommendations and suggestions throughout the day to the principal and leadership team 
members using a technique described in the literature as cycles of reflections - (compare theory 
of what an organization wants to happen with reality). During this process the school becomes 
united as a community of learners with the school staff and the Audit Panel servings as critical 
friends and coaches. 

Step 6: Taking Stock: Modified Empowerment Evaluation Session 

Empowerment Evaluation helps people align what they say they are doing with what they 
are really doing, by providing them with a continuous feedback loop designed to refine and 
improve their practice. Empowerment Evaluation is guided by many concepts including: 1) 
building a culture of evidence to make decisions; 2) using cycles of reflection—helping people 
think about their data, act on it, and then reflect on the impact of those decisions; 3) building a 
community of learners—where everyone is learning from each other along the way; and 4) 
cultivating reflective practitioners—people who think about how they can improve their 
performance on a daily basis. Guided Empowerment Self-Evaluation is also guided by a critical 
friend or coach who values the effort, but also asks hard questions to keep things rigorous and on 
track. 
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During the Empowerment Self-Audit faculty meeting, an overview of the Guided 
Empowerment Self-Audit process is provided and staff members participate in a three-step 
approach to empowerment evaluation including: 1) reviewing the mission; 2) taking stock; and 
3) planning for the future. During this process, the participants determine whether the staff are 
able to successfully articulate the school mission and vision. Additionally, time is allotted to 
assess how well the staff perceives they are doing in relation to the Nine Indicators on the 
Scholastic Audit and the Seven Principles of School Turnaround. 

This honest critique sets the stage for the faculty to plan for the future or establish new 
goals to accomplish their objectives in areas where the ratings are less than acceptable and make 
mid-course corrections to the existing School Improvement Plan. The staff specifically 
participates in the following: 

• A discussion of the mission of the school and how decisions are made based on 
the mission. 

• An open self- audit rating of the school on the nine indicators of school 
improvement outlined by the ADE by placing one of five dots on the any of the 
nine indicators that they judged to be most effective. 

• An open self-audit of themselves on the seven principles of School Turnaround 
rating by giving a rating from one to five. Five was the highest score and one was 
the lowest score. 

• The individual electronic completion of the classroom expectation and teacher 
perception surveys. 

• A discussion of the results with a focus of planning for the future or making mid¬ 
course corrections. 

• Through the use of cycles of reflections, the results were presented to the 
principal for use in school improvement planning and outcomes including the 
unexpected revelations. 

Empowerment Evaluation “Taking Stock” Activity Results 

For the Taking Stock Activity, each participant is asked to note which School 
Improvement Standards the school is demonstrating the most outstanding achievement on. 
Although there are 9 standards; each participant is asked to vote for the 5 standards he/she thinks 
are being addressed most outstandingly in the school. An example of the results from one 
school’s Taking Stock Activity is included in the table below. 


Table 3: Taking Stock 


School Improvement 

Standards 

Scholastic Audit 

Number of Participants 
voting for this Standard 

Percentage of the 

Participants voting for this 
standard. 

Standard 1: Curriculum 

24 

86% 

Standard 2: Assessment 

4 

14% 

Standard 3: Instruction 

27 

96% 

Standard 4: School Culture 

24 

86% 

Standard 5: 

Community/Family Support 

21 

75% 
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Standard 6: Professional 
Development 

4 

14% 

Standard 7 : Leadership 

21 

75% 

Standard 8: Organization 

12 

43% 

Standard 9: Comprehensive 
Planning 

3 

11% 


Note that instruction received the highest self-rating by the faculty in all eight schools. It 
is obvious the faculty in this school perceives a need for change in assessment and professional 
development activities. This information can be used to guide the school improvement process 
as it moves forward. 

An example of the results from another portion of the Taking Stock Activity is included 
in the table below. In this activity, the faculty are asked to rate their school on each of the Seven 
Turnaround Principles using a scale of 1-5 with 5 being the highest. 

Table 4: Seven Turnaround Principles Activity 


Seven Turnaround Principles 



Mean 


Principle 1: Quality Teaching 
Principle 2: Leadership 
Principle 3: Data Usage 
Principle 4: Environment 
Principle 5: Instruction 
Principle 6: Community 
Engagement 

Principle 7: Extended Day/Year 


18 

15 

22 

17 

6 


5 
8 
2 
3 

6 
12 

1 


2 

14 

1 

5 


1 

1 

7 

1 

1 

12 


1 

1 

3 


4.63 

4.48 

2.62 

4.73 

4.56 

3.79 

2.16 


Step 7: Perception Surveys 


On-line surveys are used to gather principal, teacher, student, and parent perceptions that 
are useful to schools in developing their school improvement plans for accreditation and 
standards implementation. The surveys also provide information about parent involvement and 
parent and student satisfaction with their schools. The surveys used in the Guided Empowerment 
Self- Audit measure participants’ views of the extent the school is performing in specific areas. 
To develop the survey, questions were selected which address the Scholastic Audit Indicators, 
the Principles of School Turnaround and research- based academic expectations for schooling. 
The items are all Likert scaled and modified to focus specifically on how the school is doing as a 
learning institution. Note: all stakeholders are asked to complete perceptional data surveys on¬ 
line: Principals, teachers, students, parents, and The Panel members. 

The surveys and the persons completing each one are enumerated in the list below. 
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1. The Classroom Instruction expectations Survey: Teachers complete this survey; the 
questions address teachers’ perceptions of administration’s expectations for teachers. 

2. The Teacher Survey: teachers complete this survey. It was based on the indicators of the 
Scholastic Audit Indicators. 

3. The Parent Survey: parents complete this survey. The questions address areas in the 
Scholastic Audit and School Turnaround Principles that address parent engagement and 
school to parent communication. 

4. The Student Survey: all students complete this survey. It addresses students’ perceptions 
of instruction practices and school climate indicators from both the Scholastic Audit 
Indicators and the School Turnaround principles. 

Step 8: Principal Meeting/ Cycles of Reflections 

Meetings are conducted with principals, both formal and informal, to review the activities 
of the day and to make interim recommendations based on observed activities. 

Recommendations for immediate wins are verbally provided to the principal. Additionally, the 
results of the staff empowerment process are discussed and standards that received the lowest 
ratings are explored to determine reasons for the low ratings given by staff members. 

Step 9: Parent interview and Focus Group 

A focus group meeting is held with parents and the essential question of, “How is the 
school doing as a learning institution?” are discused. 

Step 10: Teacher Interviews and Focus Group 

Throughout the day interviews are conducted relative to collecting evidence to determine 
the answer to the essential question, “How is the school doing as a learning institution?” 

DURING THE ON-SITE VISIT: DAY TWO 

Step 1: Benchmarking 

A second benchmarking procedure is completed. The process is the same as explained in 
“During the visit: Day One.” 

Step 2: Observations of Classrooms 

Classroom observations are continued with activities being noted such as the presence of 
classroom rules and procedures, learning objectives, student engagement, the level of Bloom’s 
students are engaged on, and more. 

Step 3: Observation of School Culture 

Elements of the school culture observed include the following: 
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1. Human Interactions 

2. Displays 

3. Celebration Artifacts 

4. Established routines, celebrations, and extended day meetings with parents and 
students 

Step 4: Observation of Procedures 

The following school procedures are observed: 

1. Before School Activities 

2. Transitions 

3. Lunch 

4. Recess. 

5. After School Activities 

Step 5: Photo Sessions 

For the purpose of the Guided Empowerment Self-Audit photos are taken in order to 
capture the spirit of the institution. The pictures are based in large part on the data collected, the 
classrooms observed, and the behavior of the students, parents, teachers, and administrators. 

Step 6: Formal Principal Interview 

Principal interview questions are designed to collect information and explore leadership 
qualities such as creating and sustaining a vision and mission, the capacity to build effective 
relationships, operational management skills, and job knowledge relative to improving student 
performance. 

Step 7: Planning for the Future 

After the initial staff perceptions are gathered. Cycles of reflections are conducted to 
ensure an objective assessment based on evidence or artifacts. All available performance data is 
reviewed to ensure that the results accurately describe the school’s level of performance. The 
school’s leadership team, staff and the Empowerment Self-Audit Panel Coaches, facilitates the 
nature and scope of this process. This process according to the staffs in all eight schools helps 
builds organizational and individual evaluation capacity and helps to create a sense of urgency 
that the schools must improve. Once the evaluation of data is finalized the necessary next step is 
to take action. Once strengths and weaknesses are identified, the Evaluation report is completed 
and all parties are kept informed. Priority improvement activities are initiated to inform action to 
improve student achievement and professional development with the active participation of the 
internal staff that participated in the evaluation. 

Evidence of the Benefits of the Empowerment Self-Audit 

This research report presents an amalgamation of the findings from Guided 
Empowerment Self-Audits conducted in 8 schools in 2013-2014. This research project was a 
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comparison of Empowerment Self-Audits performed by 8 schools to determine common areas of 
concern uncovered by the schools. One of the major areas of concern revealed by the Guided 
Empowerment Self-Audits was a disconnect between teachers’ perceptions of the 
implementation of the Eleven Indicators of Effective Schooling and the school’s student 
achievement data. Identifying barriers to success can aid any organization as it seeks to improve. 

All eight schools’ perceptions of their schools’ performance in the following areas 
demonstrated a miss-match between the schools’ initial perceptions and the evidence in the 
following areas: 

1. Standard 1: Curriculum, 

2. Standard 3: Instruction, and 

3. Standard 9: Comprehensive Planning. 

Provided in the table below is a general description of the miss-match between the 
stakeholders’ perceptions and the reality of analyzed data in the areas of curriculum, instruction 
and comprehensive planning in the 8 schools which participated in the Guided Empowerment 
Self-Audits. 


Table 5: Documentation of Misconceptions - Curriculum 


Indicator 

Educator’s Perception 

Actual data 

Capacity 

Academic Performance 

Generally, most schools 

Curriculum documents 

Schools 

Standard 1 

rated themselves 

in all eight schools were 

identified 

Curriculum-The school 

exceptional in this area 

under development. 

the need to 

develops and implements 

even in some cases when 

Unpacking the 

develop a 

a curriculum that is 

there was no evidence of a 

curriculum had not 

curriculum 

rigorous, intentional, and 

Rigorous, Aligned 

occurred and alignment 

that is 

aligned to state and local 

Curriculum. 

to state framework and 

rigorous, 

standards. 


Common Core State 
Standards were 
marginal. 

intentional, 

and 

aligned to 
state and 
local 

standards. 


Table 6: Documentation of Misconceptions- Instruction 


Indicator 

Educator’s Perception 

Actual Data 

Capacity 

Academic Performance 
Standard 3: Instruction- 
The school’s instructional 
program actively engages 
all students by using 
effective, varied, and 
research-based practices 
to improve student 

Generally, most schools 
rated themselves 
exceptional in this area 
even in some cases when 
there was no evidence in 
student performance or 
classroom observations 
that supported the 

All 8 schools 
participated in the 
audit process in part 
because they were 
identified as being 
in need of school 
improvement based 
on low student 

Schools 
identified the 
need to 
improve 
performance 
by using 
researched - 
based 
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academic performance 

perception 

academic 

strategies 



achievement. 

proven to 



Interviews with 

increase 



teachers also 

student 



revealed a general 
lack of 

understanding of 
effective, research- 
based teaching 
strategies. 

performance. 


Table 7: Documentation of Misconceptions- Comprehensive Planning 


Indicator 

Educator’s Perception 

Actual Data 

Capacity 

Standard 9: Comprehensive 

Generally, most 

Comprehensive plans 

Schools 

and Effective Planning-The 

schools rated 

showed little or no 

identified the 

school/district develops, 

themselves exceptional 

evidence of active 

need to improve 

implements, and evaluates 

in this area even in 

participation of 

performance by 

a comprehensive school 

some cases when there 

stakeholders, or a 

developing 

improvement plan that 

was little or no 

a clear purpose, 

comprehensive, 

communicates a clear 

evidence of 

direction, and action 

inclusive plans 

purpose, direction, and 

comprehensive 

plan focused on 

that 

action plan focused on 

planning. 

teaching and 

communicate a 

teaching and learning. 


learning. 

clear purpose, 
direction, and 
action plan 
focused on 
teaching and 
learning. 


The table below indicates the change in ratings on the Eleven Indicators of Effective 
Schooling from the first Taking Stock Activity to a subsequent analysis comparing the 
perceptions of the stakeholders with the student data and observations. 
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Table 8: Changes in Perceptions after evidence -based inquiry and school documentation 
review. 


School Improvement Standards 

Scholastic Audit rating 

Number of 
schools 
rating with a 
Superior in 
this area 

Number of 
schools with 
data to 
validate 
perceptions 

Change in rating 
after internal 
review in 
agreement with 
internal 
documentation 

Standard 1: Curriculum 

8 

2 

8 

Standard 2: Assessment 

4 

4 

4 

Standard 3: Instruction 

8 

4 

4 

Standard 4: School Culture 

8 

4 

6 

Standard 5: Community/Family Support 

4 

4 

4 

Standard 6: Professional Development 

4 

4 

4 

Standard 7 : Leadership 

6 

6 

6 

Standard 8: Organization 

4 

4 

4 

Standard 9: Comprehensive Planning 

6 

2 

8 

Standard 10: Data Use 

8 

3 

8 

Standard 11: Extended Day 

4 

4 

4 


CONCLUSIONS 


The Guided Empowerment Self-Audit Process is an effective tool for schools to use to 
foster the school improvement process. When the people who do the work are active in creating 
the plan for improvement they are more likely to carry out the plan. This research project 
indicated that the process is effective since the schools changed their own ratings on the Eleven 
Indicators of Effective Schooling after they were guided through the process of matching data 
with perception. The Guided Empowerment Self-Audit process affords the school a common 
theoretical framework, process for evaluation, and a common language to guide the process of 
planning for school improvement. 
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